Method for inspecting mounting state of component, printed circuit board inspection apparatus, and computer readable recording medium

ABSTRACT

A printed circuit board inspection apparatus can inspect the mounting state of a component by generating depth information on the component mounted on a printed circuit board by using a pattern of light reflected from the component and received by an image sensor, inputting the generated depth information into a machine-learning-based model, obtaining depth information with reduced noise from the machine-learning-based model, and using depth information with reduced noise.

TECHNICAL FIELD

The present disclosure relates to a method for inspecting a mountingstate of a component, a printed circuit board inspection apparatus, anda computer-readable recording medium.

BACKGROUND

In general, in a manufacturing process using surface-mount technology(SMT) on a printed circuit board, a screen printer prints solder pasteon the printed circuit board, and a mounter mounts components on theprinted circuit board printed with the solder paste.

In addition, an automated optical inspection (AOI) device is used toinspect the mounting state of the components mounted on the printedcircuit board. The AOI device inspects whether the components arenormally mounted on the printed circuit board without displacement,lifting, or tilting by using a captured image of the printed circuitboard.

On the other hand, in a process in which the AOI device generates animage for the printed circuit board, noise may occur in multiplereflections of light irradiated on the printed circuit board or in theprocess of processing received light by an image sensor. That is,optical noise and signal noise may variously occur, and if the noiseoccurring in this manner is not reduced, the quality of the capturedimage of the printed circuit board generated by the AOI device maydeteriorate. When the quality of the captured image of the printedcircuit board deteriorates, inspection of the mounting state of thecomponents mounted on the printed circuit board using the captured imageof the printed circuit board may not be accurately performed.

SUMMARY

The present disclosure may provide a printed circuit board inspectionapparatus that inspects a mounting state of a component by using depthinformation with reduced noise on the component obtained based on depthinformation on the component.

The present disclosure may provide a computer-readable recording mediumthat records a program including executable instructions for inspectinga mounting state of a component by using depth information with reducednoise on the component obtained based on depth information on thecomponent.

The present disclosure may provide a method of inspecting a mountingstate of a component by using depth information with reduced noiseobtained based on depth information on the component.

According to one embodiment of the present disclosure, a printed circuitboard inspection apparatus may inspect a mounting state of a componentmounted on a printed circuit board, and the printed circuit boardinspection apparatus may include: a plurality of first light sourcesconfigured to irradiate the component with a pattern of light; an imagesensor configured to receive a pattern of light reflected from thecomponent; a memory configured to store a machine-learning-based,wherein when first depth information on a first object generated using apattern of light reflected from the first object among patterns of lightirradiated from a plurality of second light sources is input into themachine-learning-based model, the machine-learning-based model outputsthe first depth information with reduced noise; and a processor, whereinthe processor generates second depth information on the component byusing the pattern of light reflected from the component and received bythe image sensor, inputs the second depth information into themachine-learning-based model, obtains the second depth information withreduced noise from the machine-learning-based model, and inspects themounting state of the component by using the second depth informationwith reduced noise.

In one embodiment, the machine-learning-based model may be trained tooutput third depth information with reduced noise by using the thirddepth information on a second object generated using a pattern of lightreflected from the second object among the patterns of light irradiatedfrom the plurality of second light sources and fourth depth informationon the second object generated using a pattern of light reflected fromthe second object among patterns of light irradiated from a plurality ofthird light sources, and may output the first depth information withreduced noise when the first depth information is input based on atraining result.

In one embodiment, the number of the plurality of second light sourcesmay be the same as the number of the plurality of first light sources,and the number of the plurality of third light sources may be largerthan the number of the plurality of first light sources.

In one embodiment, the machine-learning-based model may include aconvolutional neural network (CNN) or a generative adversarial network(GAN).

In one embodiment, the processor may generate a three-dimensional imageof the second component by using the second depth information withreduced noise, and may inspect the mounting state of the component byusing the three-dimensional image of the second component.

In one embodiment, when visibility information on the first object isfurther input into the machine-learning-based model, themachine-learning-based model may output the first depth information withreduced noise by using the visibility information.

According to one embodiment, the machine-learning-based model may betrained to output third depth information with reduced noise using thirddepth information on a second object, which is generated using a patternof light reflected from the second object among the pattern of lightradiated from the plurality of second light sources, visibilityinformation on the second object, which is generated using the patternof light reflected from the second object among the patterns of lightradiated from the plurality of second light sources, and fourth depthinformation on the second object, which is generated using a pattern oflight reflected from the second object among patterns of light radiatedfrom a plurality of third light sources, and may output the first depthinformation with reduced noise based on a training result when the firstdepth information and visibility information on the component are input.

In one embodiment, the processor may generate visibility information onthe component by using the pattern of light reflected from the componentand received by the image sensor, and may further input the visibilityinformation on the component into the machine-learning-based model.

According to one embodiment of the present disclosure, a printed circuitboard inspection apparatus for inspecting a mounting state of acomponent mounted on a printed circuit board may include: a plurality offirst light sources to radiate a component with a pattern of light; animage sensor to receive the pattern of light reflected from thecomponent; a memory to store a machine-learning-based model, wherein,wherein when a plurality of pieces of depth information on a firstobject, which is generated using a pattern of light reflected from thefirst object among patterns of light emitted from a plurality of secondlight sources, is input into the machine-learning-based model, themachine-learning-based model generates first depth information andoutputs the first depth information with reduced noise; and a processor,wherein the processor may generate a plurality of pieces of depthinformation on the component using the pattern of light which isreflected from the component and is received by the image sensor, mayinput the plurality of pieces of depth information on the component tothe machine-learning-based model, may obtain second depth informationwith reduced noise from the machine-learning-based model, in whichsecond depth information is generated by the machine-learning-basedmodel based on the plurality of pieces of depth information on thecomponent, and may inspect the mounting state of the component using thesecond depth information with reduced noise.

According to one embodiment, the machine-learning-based model may betrained to generate and output third depth information with reducednoise using third depth information, which is generated using aplurality of pieces of depth information on the second object generatedusing a pattern of light reflected from the second object among thepatterns of light emitted from the plurality of second light sources,and fourth depth information on the second object, wherein themachine-learning-based model is generated using a pattern of lightreflected from the second object among patterns of light radiated from aplurality of third light sources, and wherein the machine-learning-basedmodel generates the first depth information and outputs the first depthinformation with reduced noise based on a training result when theplurality of pieces of depth information on the first object is input.

According to one embodiment, the plurality of second light sources maybe the same in number as the plurality of first light sources, and theplurality of third light sources may be greater in number than theplurality of first light sources.

According to one embodiment of the present disclosure, a non-transitorycomputer-readable recording medium may record a program to be performedon a computer, wherein the program includes executable instructions thatcauses, when executed by a processor, the processor to performoperations of: controlling a plurality of first light sources toirradiate a component mounted on a printed circuit board with a patternof light; generating first depth information on the component by usingthe pattern of light reflected from the component and received by animage sensor; inputting the first depth information into amachine-learning-based model; obtaining first depth information withreduced noise on the component from the machine-learning-based model;and inspecting the mounting state of the component by using the firstdepth information with reduced noise on the component, and wherein, whenfirst depth information on the first object generated using a pattern oflight reflected from the first object among patterns of light irradiatedfrom a plurality of second light sources is input, themachine-learning-based model outputs the first depth information withreduced noise.

In one embodiment, the machine-learning-based model may be trained tooutput third depth information with reduced noise by using the thirddepth information on the second object generated using the pattern oflight reflected from a second object among the patterns of lightirradiated from the plurality of second light sources and fourth depthinformation on the second object generated using a pattern of lightreflected from the second object among patterns of light irradiated froma plurality of third light sources, and may output the first depthinformation with reduced noise when the first depth information is inputbased on a training result.

In one embodiment, in the machine-learning-based model, the number ofthe plurality of second light sources may be the same as the number ofthe plurality of first light sources, and the number of the plurality ofthird light sources may be larger than the number of the plurality offirst light sources.

In one embodiment, the machine-learning-based model may include aconvolutional neural network (CNN) or a generative adversarial network(GAN).

In one embodiment, the executable instructions may cause the processorto further perform operations of: generating a three-dimensional imageof a second component by using second depth information with reducednoise; and inspecting the mounting state of the component by using thethree-dimensional image of the second component.

In one embodiment, when visibility information on the first object isfurther input, the machine-learning-based model may output the firstdepth information with reduced noise by using the visibilityinformation.

In one embodiment, the executable instructions may cause the processorto further perform operations of: generating visibility information onthe component by using the pattern of light reflected from the componentand received by the image sensor, and further inputting the visibilityinformation into the machine-learning-based model.

According to one embodiment of the present disclosure, a method ofinspecting a mounting state of a component by a printed circuit boardinspection apparatus may include: controlling a plurality of first lightsources to irradiate a component mounted on a printed circuit board witha pattern of light; generating first depth information on the componentby using the pattern of light reflected from the component and receivedby an image sensor; inputting the first depth information into amachine-learning-based model; obtaining the first depth information withreduced noise on the component from the machine-learning-based model;and inspecting the mounting state of the component by using the firstdepth information with reduced noise on the component, and wherein, whenfirst depth information on the first object generated using a pattern oflight reflected from the first object among patterns of light irradiatedfrom a plurality of second light sources is input, themachine-learning-based model outputs the first depth information withreduced noise.

In one embodiment, the machine-learning-based model may be trained tooutput third depth information with reduced noise by using the thirddepth information on the second object generated using the pattern oflight reflected from a second object among the patterns of lightirradiated from the plurality of second light sources and fourth depthinformation on the second object generated using a pattern of lightreflected from the second object among patterns of light irradiated froma plurality of third light sources, and may output the first depthinformation with reduced noise when the first depth information is inputbased on a training result.

As described above, the printed circuit board inspection apparatusaccording to various embodiments of the present disclosure may processthe depth information on the component through themachine-learning-based model, thereby reducing noise from the depthinformation on the component and inspecting the mounting state of thecomponent mounted on the printed circuit board by using the depthinformation with reduced noise on the component. The printed circuitboard inspection apparatus may remove noise such as an unreceived signalor a peak signal from the depth information on the component by usingthe machine-learning-based model even though a relatively small numberof pieces of image data are obtained to generate the depth information,and may generate the depth information on the component so that the lostshape can be restored using the machine-learning-based model even thougha relatively small number of pieces of image data are obtained so thatinformation for generating the depth information is insufficient. Inaddition, the printed circuit board inspection apparatus may not performerror restoration of the joint shape of the component while correctingthe three-dimensional (3D) sharpness of the edges of the component asmuch as possible, and may detect the shape of an additionally measuredforeign material without deteriorating the same.

In this manner, by reducing noise in the depth information on thecomponent and by performing shape restoration on the component mountedon the printed circuit board and a solder paste as closely as possibleto the shape of the actual component and solder paste, it is possible toinspect the mounting state of the component more accurately.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates a printed circuit board inspection apparatusaccording to various embodiments of the present disclosure.

FIG. 2 is a block diagram illustrating a printed circuit boardinspection apparatus according to various embodiments of the presentdisclosure.

FIG. 3 is a flowchart illustrating a method of inspecting a mountingstate of a component by a printed circuit board inspection apparatusaccording to various embodiments of the present disclosure.

FIGS. 4A to 4C are conceptual diagrams illustrating a learning method ofa machine-learning-based model according to various embodiments of thepresent disclosure.

FIGS. 5A to 5C are conceptual diagrams illustrating the operation of amachine-learning-based model according to various embodiments of thepresent disclosure.

FIG. 6 is a conceptual diagram illustrating a learning method of amachine-learning-based model according to various embodiments of thepresent disclosure.

FIG. 7 is a diagram illustrating a method of acquiring depth informationused in training a machine-learning-based model according to variousembodiments of the present disclosure.

FIGS. 8A to 8C illustrate images of a component generated using depthinformation with reduced noise by a printed circuit board inspectionapparatus according to various embodiments of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure are illustrated for describing thetechnical spirit of the present disclosure. The scope of the claimsaccording to the present disclosure is not limited to the embodimentsdescribed below or to the detailed descriptions of these embodiments.

All technical or scientific terms used herein have meanings that aregenerally understood by a person having ordinary knowledge in the art towhich the present disclosure pertains, unless otherwise specified. Theterms used herein are selected for only more clear illustration of thepresent disclosure, and are not intended to limit the scope of claims inaccordance with the present disclosure.

The expressions “include”, “provided with”, “have” and the like usedherein should be understood as open-ended terms connoting thepossibility of inclusion of other embodiments, unless otherwisementioned in a phrase or sentence including the expressions.

A singular expression can include meanings of plurality, unlessotherwise mentioned, and the same is applied to a singular expressionstated in the claims.

The terms “first”, “second”, etc., used herein are used to identify aplurality of components from one another, and are not intended to limitthe order or importance of the relevant components.

The term “unit” used in these embodiments means a software component orhardware component, such as a field-programmable gate array (FPGA) andan application specific integrated circuit (ASIC). However, a “unit” isnot limited to software and hardware, it may be configured to be anaddressable storage medium or may be configured to run on one or moreprocessors. For example, a “unit” may include components, such assoftware components, object-oriented software components, classcomponents, and task components, as well as processors, functions,attributes, procedures, subroutines, segments of program codes, drivers,firmware, micro-codes, circuits, data, databases, data structures,tables, arrays, and variables. Functions provided in components and“units” may be combined into a smaller number of components and “units”or further subdivided into additional components and “units.”

The expression “based on” used herein is used to describe one or morefactors that influence a decision, an action of judgment or an operationdescribed in a phrase or sentence including the relevant expression, andthis expression does not exclude additional factor influencing thedecision, the action of judgment or the operation.

When a certain component is described as “coupled to” or “connected to”another component, this should be understood as having meaning that thecertain component may be coupled or connected directly to the othercomponent or that the certain component may be coupled or connected tothe other component via a new intervening component.

Hereinafter, embodiments of the present disclosure will be describedwith reference to the accompanying drawings. In the accompanyingdrawings, like or relevant components are indicated by like referencenumerals. In the following description of embodiments, repeateddescriptions of the identical or relevant components will be omitted.However, even if a description of a component is omitted, such acomponent is not intended to be excluded in an embodiment.

FIG. 1 illustrates a printed circuit board inspection apparatusaccording to various embodiments of the present disclosure.

According to various embodiments of the present disclosure, a printedcircuit board inspection apparatus 100 may inspect the mounting state ofat least one component mounted on a printed circuit board 110. Atransport unit 120 may move the printed circuit board 110 to apredetermined position in order to inspect the mounting state of thecomponents. In addition, when the inspection is completed by the printedcircuit board inspection apparatus 100, the transport unit 120 may movethe printed circuit board 110, which has been inspected, to deviate fromthe predetermined position, and may move another printed circuit board111 to a predetermined printed circuit board.

According to various embodiments of the present disclosure, the printedcircuit board inspection apparatus 100 may include a first light source101, an image sensor 102, and a frame 103. The first light source 101and the image sensor 102 may be fixed to the frame 103. The number andarrangement of each of the first light source 101, the image sensor 102,and the frame 103 shown in FIG. 1 are for the purpose of explanation,but are not limited thereto. For example, one first light source 101 maybe arranged in the position of the image sensor 102 shown in FIG. 1, anda plurality of image sensors may be arranged in the position of thefirst light source 101 shown in FIG. 1. The first light source 101 andthe image sensor 102 may be arranged in various directions and anglesthrough the plurality of frames 103.

In one embodiment, the first light source 101 may irradiate, with apattern of light, the printed circuit board 110 moved to a predeterminedposition to inspect the mounting state of the component. In the case ofa plurality of first light sources 101, they may be arranged to havedifferent irradiation directions, different irradiation angles, and thelike. In addition, in the case of a plurality of first light sources101, pitch intervals of the pattern of light irradiated from the firstlight sources 101 may be different from each other. For example, thepattern of light may be light having a pattern having a certain period,which is irradiated to measure a three-dimensional (3D) shape of theprinted circuit board 110. The first light source 101 may irradiate apattern of light in which the brightness of the stripes has a sine waveshape, a pattern of light in an on-off form in which bright and darkparts are repeatedly displayed, or a triangular wave-pattern of lighthaving a triangular waveform with a change in brightness. However, thisis for illustrative purposes only, and the present disclosure is notlimited thereto, and the first light source 101 may irradiate lightincluding various types of patterns in which a change in brightness isrepeated at a constant period.

In one embodiment, the image sensor 102 may receive a pattern of lightreflected from the printed circuit board 110 and the component mountedon the printed circuit board 110. The image sensor 102 may generateimage data using the received pattern of light.

FIG. 2 is a block diagram illustrating a printed circuit boardinspection apparatus according to various embodiments of the presentdisclosure.

According to various embodiments of the present disclosure, the printedcircuit board inspection apparatus 100 may include a first light source210, an image sensor 220, a memory 230, and a processor 240. Inaddition, the printed circuit board inspection apparatus 100 may furtherinclude a communication circuit 250. Each component included in theprinted circuit board inspection apparatus 100 may be electricallyconnected to each other to transmit and receive signals and data.

In one embodiment, the printed circuit board inspection apparatus 100may include a plurality of first light sources 210. The first lightsource 210 may irradiate an inspection object (e.g., a printed circuitboard) with a pattern of light. For example, the first light source 210may irradiate the entire inspection object with a pattern of light ormay irradiate an object (e.g., a component mounted on a printed circuitboard) included in the inspection object with a pattern of light.Hereinafter, for convenience of description, although the first lightsource 210 is mainly described as irradiating the component mounted onthe printed circuit board with a pattern of light, the presentdisclosure is not limited thereto. The first light source 210 mayirradiate, with a pattern of light, the entire printed circuit board tobe inspected or one region of the printed circuit board including atleast one component mounted on the printed circuit board.

In one embodiment, the first light source 210 may include a light source(not shown), a grating (not shown), a grating transport device (notshown), and a projection lens unit (not shown). The grating can convertlight irradiated from the light source into a pattern of light. Thegrating can be transported through a grating transport mechanism, forexample a piezo actuator (PZT), to generate phase-shifted pattern oflight. The projection lens unit may allow the pattern of light generatedby the grating to be irradiated to the component mounted on the printedcircuit board, which is an object included in the inspection object.Further, the first light source 210 may form a pattern of light throughvarious methods such as liquid crystal display (LCD), digital lightprocessing (DLP), and liquid crystal on silicon (LCOS), and may allowthe formed pattern of light to be irradiated to the component mounted onthe printed circuit board which is an object included in the inspectionobject.

In one embodiment, the image sensor 220 may receive a pattern of lightreflected from the component. For example, the image sensor 220 mayreceive the pattern of light reflected from the component to generateimage data on the component. The first image sensor 220 may transmit thegenerated image data on the component to the processor 240.

In one embodiment, the memory 230 may store instructions or data relatedto at least one other component of the printed circuit board inspectionapparatus 100. Also, the memory 230 may store software and/or programs.For example, the memory 230 may include an internal memory or anexternal memory. The internal memory may include at least one ofvolatile memory (e.g., DRAM, SRAM or SDRAM), and non-volatile memory(e.g., flash memory, hard drive, or solid state drive (SSD)). Theexternal memory may be functionally or physically connected to theprinted circuit board inspection apparatus 100 through variousinterfaces.

In one embodiment, the memory 230 may store instructions for operatingthe processor 240. For example, the memory 230 may store instructionsthat cause the processor 240 to control other components of the printedcircuit board inspection apparatus 100 and to interwork with an externalelectronic device or a server. The processor 240 may control the othercomponents of the printed circuit board inspection apparatus 100 basedon the instructions stored in the memory 230 and may interwork with theexternal electronic device or the server. Hereinafter, the operation ofthe printed circuit board inspection apparatus 100 will be describedmainly with each component of the printed circuit board inspectionapparatus 100. Also, instructions for performing an operation by eachcomponent may be stored in the memory 230.

In one embodiment, the memory 230 may store a machine-learning-basedmodel. The machine-learning-based model may receive first depthinformation on a first object using a pattern of light reflected fromthe first object among patterns of light irradiated from a plurality ofsecond light sources. For example, the first depth information mayinclude at least one of a shape, color information for each pixel,brightness information, and a height value.

For example, the plurality of second light sources and the plurality offirst light sources 210 may be the same or different. Although theplurality of second light sources are different from the plurality offirst light sources 210, the number of the plurality of second lightsources may be the same as the number of the plurality of first lightsources 210. Further, even if the plurality of second light sources areincluded in another printed circuit board inspection apparatus, thearrangement positions of the plurality of second light sources in theother printed circuit board inspection apparatus may correspond to thearrangement positions of the plurality of first light sources in theprinted circuit board inspection apparatus 100. In themachine-learning-based model, when the first depth information is input,first depth information with reduced noise may be output.

For example, the first depth information generated using a pattern oflight reflected from the first object may generate noise in multiplereflections of the pattern of light irradiated on the first object or inthe process of processing the received light by the image sensor. Forexample, the noise may be a portion of the first depth information thatdoes not correspond to the shape of the first object or that isdetermined not to be related to the first object. In order to improvethe quality of the image on the first object, for example, the 3D imageon the first object, the machine-learning-based model may be trained tooutput the first depth information with reduced noise. Examples of themachine-learning-based model may include a convolutional neural network(CNN), a generative adversarial network (GAN), and the like. When thefirst depth information is input to the machine-learning-based model, adetailed method of training the machine-learning-based model to outputthe first depth information with reduced noise will be described later.

In addition, the machine-learning-based model may be stored in a memoryof an external electronic device or server interworking with the printedcircuit board inspection apparatus 100 by wire or wirelessly. In thiscase, the printed circuit board inspection apparatus 100 may transmitand receive information to and from the external electronic device orserver interworked by wire or wirelessly to reduce the noise of thefirst depth information.

In one embodiment, the processor 240 may drive an operating system or anapplication program to control at least one other component of theprinted circuit board inspection apparatus 100, and may perform avariety of data processing, calculation, and the like. For example, theprocessor 240 may include a central processing unit or the like, or maybe implemented as a system on chip (SoC).

In one embodiment, the communication circuit 250 may communicate with anexternal electronic device or an external server. For example, thecommunication circuit 250 may establish communication between theprinted circuit board inspection apparatus 100 and an externalelectronic device. The communication circuit 250 may be connected to anetwork through wireless communication or wired communication tocommunicate with an external electronic device or external server. Asanother example, the communication circuit 250 may be connected to anexternal electronic device in a wired manner to perform communication.

The wireless communication may include, for example, cellularcommunication (e.g., LTE, LTE advance (LTE-A), code division multipleaccess (CDMA), wideband CDMA (WCDMA), universal mobiletelecommunications system (UMTS), wireless broadband (WiBro), etc.).Further, the wireless communication may include short-range wirelesscommunication (e.g., Wi-Fi, light fidelity (Li-Fi), Bluetooth, Bluetoothlow power (BLE), Zigbee, near field communication (NFC), etc.).

In one embodiment, the processor 240 may generate second depthinformation on the component by using the pattern of light reflectedfrom the component mounted on the printed circuit board received by theimage sensor 220. For example, the processor 240 may generate seconddepth information on the component by using an image of the componentgenerated using the pattern of light reflected from the componentgenerated by the image sensor 220. As another example, the image sensor220 may transmit the received information on the pattern of light to theprocessor 240, and the processor 240 may generate an image of thecomponent and may use the image of the component to generate the seconddepth information on the component. The processor 240 may generate thesecond depth information on the component by applying an opticaltriangulation method or a bucket algorithm to the image of thecomponent. However, this is for illustrative purposes only, and thepresent disclosure is not limited thereto, and the second depthinformation on the component may be generated through various methods.

In one embodiment, the processor 240 may input the second depthinformation to the machine-learning-based model. For example, when themachine-learning-based model is stored in the memory 230, the processor240 may directly input the second depth information to themachine-learning-based model. As another example, when themachine-learning-based model is stored in an external electronic deviceor an external server, the processor 240 may control the communicationcircuit 250 to transmit the second depth information to the externalelectronic device or the external server.

In one embodiment, the processor 240 may obtain the second depthinformation with reduced noise from the machine-learning based model.For example, when the machine-learning-based model is stored in thememory 230, the processor 240 may input the second depth informationwith reduced noise directly from the machine-learning-based model. Asanother example, when the machine-learning based model is stored in theexternal electronic device or the external server, the processor 240 mayobtain the second depth information with reduced noise from the externalelectronic device or the external server through the communicationcircuit 250.

In one embodiment, the processor 240 may inspect the mounting state ofthe component mounted on the printed circuit board by using the seconddepth information with reduced noise. For example, the processor 240 maygenerate a 3D image of the component using the second depth informationwith reduced noise. In addition, the processor 240 may inspect themounting state of the component using the generated 3D image of thecomponent. For example, the processor 240 may use the 3D image of thecomponent to inspect whether the component is mounted at a predeterminedposition, whether the component is mounted in a predetermined direction,whether at least a portion of the component is tilted and mounted,whether there is a foreign object in the component, or the like, therebyinspecting the mounting state of the component.

In one embodiment, when visibility information about the first object isfurther input, the machine-learning-based model may output first depthinformation with reduced noise using the visibility information. Forexample, the visibility information is information indicating the degreeof noise, and the machine-learning-based model may use the visibilityinformation to effectively reduce the noise in the first depthinformation. A specific method for training the machine-learning-basedmodel to output the first depth information with reduced noise when thefirst depth information and the visibility information are input to themachine-learning-based model will be described later.

In one embodiment, the processor 240 may generate visibility informationabout a component using a pattern of light which is reflected from thecomponent and is received by the image sensor 220. For example,visibility information represents the ratio of the amplitude (Bi(x, y))of a brightness signal of image data to an average brightness (Ai(x, y))and tends to generally increase as reflectivity increases. Thevisibility information (Vi(x, y)) may be represented by Equation 1.V _(i)(x,y)=B _(i)(x,y)/A _(i)(x,y)  [Equation 1]

For example, patterns of light may be emitted respective from theplurality of first light sources 210 to the printed circuit board invarious directions, thereby generating a plurality of pieces of imagedata of a component by the image sensor 220 or the processor 240. Theprocessor 240 may extract N brightness degrees (Ii1, Ii2, . . . , andIiN) at individual positions (i(x, y)) in an X-Y coordinate system fromthe plurality of pieces of generated image data and may calculate anaverage brightness (Ai(x, y)) using an amplitude (Bi(x, y)) and anN-bucket algorithm. The processor 240 may generate visibilityinformation (Vi(x,y)) using the calculated amplitude (Bi(x, y)) andaverage brightness (Ai(x, y)). In addition, the processor 240 mayfurther input the generated visibility information about the componentinto the machine-learning-based model.

In one embodiment, the machine-learning-based model may receive aplurality of pieces of depth information about a first object generatedusing a pattern of light reflected from the first object among patternsof light emitted from the plurality of second light sources. Since eachof the plurality of second light sources emits a pattern of light to thefirst object and the pattern of light emitted by each of the pluralityof second light sources is reflected from the first object and isreceived by the image sensor, a plurality of pieces of depth informationabout the first object may be generated.

When the plurality of pieces of depth information is input, themachine-learning-based model may generate and output first depthinformation with reduced noise. A specific method for training themachine-learning-based model to generate and output the first depthinformation with reduced noise when the plurality of pieces of depthinformation is input to the machine-learning-based model will bedescribed later. For example, the first depth information isrepresentative depth information about the first object and may begenerated based on the plurality of pieces of depth information aboutthe first object.

In one embodiment, the processor 240 may generate a plurality of piecesof depth information about a component using a pattern of light which isreflected from the component and is received by the image sensor 220.Since each of the plurality of first light sources emits a pattern oflight to the component and the pattern of light emitted by each of theplurality of first light sources is reflected from the component and isreceived by the image sensor 220, a plurality of pieces of depthinformation about the component may be generated.

The processor 240 may input the plurality of pieces of depth informationabout the component into the machine-learning-based model. For example,each of the plurality of first light sources 210 emits a pattern oflight to the component mounted on the printed circuit board, and theimage sensor 220 may generate a plurality of pieces of image data aboutthe component using the pattern of light reflected from the component.The image sensor 220 may transmit the plurality of pieces of image datato the processor 240. The processor 240 may generate a plurality ofpieces of depth information about the component using the plurality ofpieces of image data.

In one embodiment, the processor 240 may obtain second depth informationwith reduced noise from the machine-learning-based model. The seconddepth information may be generated by the machine-learning-based modelbased on the plurality of pieces of depth information about thecomponent. For example, the second depth information is representativedepth information about the component and may be generated based on theplurality of pieces of depth information about the component.

In one embodiment, the machine-learning-based model may receive aplurality of pieces of image data about a first object generated using apattern of light reflected from the first object among patterns of lightemitted from the plurality of second light sources. When the pluralityof pieces of image data is input, the machine-learning-based model maygenerate and output first depth information with reduced noise. Aspecific method for training the machine-learning-based model togenerate and output the first depth information with reduced noise whenthe plurality of pieces of image data about the first object is input tothe machine-learning-based model will be described later.

In one embodiment, the processor 240 may input a plurality of pieces ofimage data about a component generated using a pattern of light which isreflected from the component and is received by the image sensor 220into the machine-learning-based model. In another example, the processor240 may generate a plurality of pieces of image data about a componentusing information about a pattern of light which is reflected from thecomponent and is received by the image sensor 220 and may input theplurality of pieces of generated image data into themachine-learning-based model.

In one embodiment, the processor 240 may obtain first depth informationwith reduced noise from the machine-learning-based model. Second depthinformation may be generated by the machine-learning-based model basedon the plurality of pieces of image data.

FIG. 3 is a flowchart illustrating a method of inspecting a mountingstate of a component by a printed circuit board inspection apparatusaccording to various embodiments of the present disclosure.

Although process steps, method steps, algorithms, and the like have beendescribed in a sequential order in the flowchart shown in FIG. 3, suchprocesses, methods, and algorithms may be configured to be operated inarbitrary appropriate orders. In other words, the steps of theprocesses, methods, and algorithms described in various embodiments ofthe present disclosure need not be performed in the order described inthis disclosure. Also, although some steps are described as beingperformed asynchronously, in other embodiments, some of these steps maybe performed simultaneously. Also, the illustration of the process bydepiction in the drawings does not mean that the illustrated processexcludes other changes and modifications thereto, that any of theillustrated processes or steps thereof is essential to one or more ofthe various embodiments of the present disclosure, or that theillustrated process is preferred.

In operation 310, the printed circuit board inspection apparatus 100 mayirradiate a component mounted on a printed circuit board with a patternof light. For example, the processor of the printed circuit boardinspection apparatus 100 may control a plurality of first light sourcessuch that the pattern of light is irradiated to each of a plurality ofcomponents mounted on the printed circuit board to be inspected.

In operation 320, the printed circuit board inspection apparatus 100 mayreceive the pattern of light reflected from the component and maygenerate second depth information on the component using the pattern oflight. For example, the first image sensor may generate an image of thecomponent using the pattern of light reflected from the component, andmay transmit the generated image of the component to the processor. Theprocessor may generate the second depth information on the componentusing the image of the component and received from the first imagesensor.

In operation 330, the printed circuit board inspection apparatus 100 mayinput the second depth information to a machine-learning-based model.For example, when the machine-learning-based model is stored in thememory of the printed circuit board inspection apparatus 100, theprocessor may directly input the second depth information to themachine-learning-based model. As another example, when themachine-learning-based model is stored in an external electronic deviceor an external server, the processor may control a communication circuitto transmit the second depth information to the external electronicdevice or the external server.

In operation 340, the printed circuit board inspection apparatus 100 mayobtain the second depth information with reduced noise from themachine-learning based model. For example, when themachine-learning-based model is stored in the memory, the processor mayobtain the second depth information with reduced noise directly from themachine-learning-based model. As another example, when themachine-learning-based model is stored in the external electronic deviceor the external server, the processor may obtain the second depthinformation with reduced noise from the external electronic device orthe external server through the communication circuit.

In operation 350, the printed circuit board inspection apparatus mayinspect the mounting state of the component using the second depthinformation with reduced noise. For example, the processor may generatea 3D image of the component using the second depth information withreduced noise. In addition, the processor may inspect the mounting stateof the component using the generated 3D image of the component.

FIGS. 4A to 4C are conceptual diagrams illustrating a learning method ofa machine-learning-based model according to various embodiments of thepresent disclosure.

Referring to FIG. 4A, a machine-learning-based model 410 may be trainedto output third depth information with reduced noise 413, using thirddepth information 411 on a second object generated using a pattern oflight reflected from the second object among patterns of lightirradiated from a plurality of third light sources and fourth depthinformation 412 on the second object generated using the pattern oflight reflected from the second object among the patterns of lightirradiated from a plurality of third light sources.

Based on results learned to output the third depth information withreduced noise 413, the machine-learning-based model 410 may output firstdepth information with reduced noise even when the first depthinformation on the first object different from the second object usedfor learning is input.

In one embodiment, the third depth information 411 and the fourth depthinformation 412 may be input to the machine-learning-based model 410 forlearning. For example, the number of the plurality of third lightsources irradiating the pattern of light used to generate the fourthdepth information 412 may be larger than the number of a plurality offirst light sources and larger than the number of a plurality of secondlight sources having the same number as the number of the plurality offirst light sources. Since the number of the plurality of third lightsources is larger than the number of the plurality of second lightsources, the number of a plurality of images of the second object usedin generating the fourth depth information 412 may be larger than thenumber of a plurality of images of the second object used in generatingthe third depth information 411. Since the irradiation direction,irradiation angle, and pitch interval of each of the plurality of fourthlight sources are different from each other, all of the plurality ofimages of the second object used in generating the fourth depthinformation 412 may be images of the second object, but they may bedifferent images from each other. Similarly, since the irradiationdirection, irradiation angle, and pitch interval of each of theplurality of third light sources are different from each other, all ofthe plurality of images of the second object used in generating thethird depth information 411 may be images of the second object, but theymay be different images from each other.

In addition, since the number of the plurality of fourth light sourcesis larger than the number of the plurality of third light sources, theplurality of fourth light sources may irradiate the second object withlight while having at least one irradiation direction, at least oneirradiation angle, and at least one pitch interval which are differentfrom those of the plurality of third light sources. Accordingly, thenumber of the plurality of images of the second object used ingenerating the fourth depth information 412 may be larger than thenumber of the plurality of images of the second object used ingenerating the third depth information 411. As a result, the generatedfourth depth information 412 may generate relatively less noise than thethird depth information 411. Accordingly, the shape of the objectmeasured through depth information generated using a large number oflight sources may be closer to the actual shape of the object comparedto the shape of the object measured through depth information generatedusing a small number of light sources.

In one embodiment, since the fourth depth information 412 generatesrelatively less noise than the third depth information 411, the fourthdepth information 412 may be used as depth information that is areference in a process in which the machine-learning-based model 410transforms the third depth information 411 to reduce noise from thethird depth information 411 or a process in which themachine-learning-based model 410 is trained to output noise from thethird depth information 411.

In one embodiment, the machine-learning-based model 410 may be trainedto transform the third depth information 411 to converge to the fourthdepth information 412. Hereinafter, for convenience of description, thethird depth information 411 transformed to converge to the fourth depthinformation 412 is referred to as transformation depth information. Forexample, the machine-learning-based model 410 may compare thetransformation depth information and the fourth depth information 412.The machine-learning-based model 410 may adjust a parameter fortransformation of the third depth information 411 based on thecomparison result. By repeating the above process, themachine-learning-based model 410 may determine the parameter fortransformation of the third depth information 411 such that the thirddepth information 411 converges to the fourth depth information 412.Through this, the machine-learning-based model 410 may be trained totransform the third depth information 411 to converge to the fourthdepth information 412. The machine-learning-based model 410 may betrained to output the transformation depth information as the thirddepth information with reduced noise 414. In this manner, themachine-learning-based model 410 may be trained to transform the thirddepth information 411 to converge to the fourth depth information 412,so that the shape of the object can be measured more accurately evenwhen the number of images of an object available in generating depthinformation is relatively insufficient.

In one embodiment, the machine-learning-based model 410 may be trainedto detect noise from the third depth information 411. For example, themachine-learning-based model 410 may be trained to detect noise from thethird depth information 411 and to output the third depth informationwith reduced noise 414 by reducing the detected noise.

For example, the machine-learning-based model 410 may be trained todetect a first portion which is determined to be noise from the thirddepth information 411, by comparing the transformation depth informationand the third depth information 411. For example, themachine-learning-based model 410 may be trained to detect a portion inwhich the difference between the transformation depth information andthe third depth information 411 is equal to or larger than apredetermined threshold, as a first portion. The machine-learning-basedmodel 410 may be trained to output the third depth information withreduced noise 413 by reducing the noise detected from the third depthinformation 411.

Referring to FIG. 4B, the machine-learning-based model 420 may betrained to output the third depth information with reduced noise 422,using the third depth information 411, the fourth depth information 412,and visibility information 421 on the second object generated using apattern of light reflected from the second object among patterns oflight irradiated from the plurality of second light sources.

The machine-learning-based model 420 may output the first depthinformation with reduced noise even if the first depth information onthe first object different from the second object used in learning andvisibility information on the first object are input, based on resultstrained to output the third depth information with reduced noise 413.

In one embodiment, the third depth information 411, the fourth depthinformation 412, and the visibility information 421 may be further inputto the machine-learning-based model 420. The machine-learning-basedmodel 420 may be trained to adjust the transformation depth informationby using the visibility information 421 to more accurately represent theshape of the second object. For example, the visibility information 421is information indicating the degree of noise occurring in the thirddepth information 411, which is depth information about the secondobject, and may indicate whether the third depth information 411 is aquality measurement value. For example, the machine-learning-based model420 may be trained to determine whether there is a second portion havinga preset threshold or greater in the visibility information 421.

For example, when the second portion exists, the machine-learning-basedmodel 420 may be trained to determine a portion of the transformationdepth information corresponding to the second portion and to adjust thepart corresponding to the second portion based on the visibilityinformation 421. The machine-learning-based model 420 may be trained tooutput the adjusted transformation depth information as the third depthinformation with reduced noise 422.

In another example, when no second portion exists, themachine-learning-based model 420 may be trained to determine not toadjust the transformation depth information and to output thetransformation depth information as the third depth information withreduced noise 422.

In one embodiment, in order to more accurately detect noise, themachine-learning-based model 420 may be trained to detect a thirdportion that is determined to be noise although it is not actually noisefrom the first part determined to be noise, using the visibilityinformation 421. When the third portion is detected, themachine-learning-based model 420 may be trained to exclude the thirdportion from the first portion and to determine the first portion fromwhich the third portion is excluded to be the noise from the third depthinformation 411. Also, when the third portion is not detected, themachine-learning-based model 420 may be trained to output the thirddepth information with reduced noise 422 by determining the firstportion to be the noise in the third depth information 411 and reducingthe noise determined in the third depth information 411.

Referring to FIG. 4C, a machine-learning-based model 430 may be trainedto generate the third depth information 411 (refer to FIG. 4A) about thesecond object using a plurality of pieces of depth information 431, 432,433, and 434 about the second object generated using a pattern of lightreflected from the second object among patterns of light emitted from aplurality of second light sources. The machine-learning-based model 430may be trained to output third depth information with reduced noise 435using the generated third depth information 411 and fourth depthinformation 412. A specific method for training themachine-learning-based model 430 to output the third depth informationwith reduced noise 435 using the third depth information 411 and thefourth depth information 412 is the same as that illustrated FIG. 4A,and thus a description thereof is omitted.

In one embodiment, the plurality of pieces of depth information 431,432, 433, and 434 about the second object generated using the pattern oflight reflected from the second object among the patterns of lightemitted from the plurality of second light sources may be input to themachine-learning-based model 430. The machine-learning-based model 430may be trained to generate the third depth information 411 (refer toFIG. 4A), which is representative depth information about the secondobject, using the plurality of pieces of depth information 431, 432,433, and 434.

Further, although not shown, a plurality of pieces of image data aboutthe second object generated using the pattern of light reflected fromthe second object among the patterns of light emitted from the pluralityof second light sources may be input to the machine-learning-based model430. The machine-learning-based model 430 may be trained to generate thethird depth information 411 (refer to FIG. 4A) about the second objectusing the plurality of pieces of image data about the second object.

In one embodiment, a plurality of pieces of image data about the secondobject generated using the pattern of light reflected from the secondobject among the patterns of light emitted from the plurality of secondlight sources may be input to the machine-learning-based model 430. Themachine-learning-based model 430 may be trained to generate the thirddepth information 411 (refer to FIG. 4A) using the plurality of piecesof image data.

In one embodiment, the fourth depth information 413 generated using apattern of light irradiated from a plurality of fourth light sources maybe generated by the printed circuit board inspection apparatus includingthe number of plurality of fourth light sources larger than the numberof plurality of third light sources. In addition, the fourth depthinformation 412 may be generated by the printed circuit board inspectionapparatus including the number of plurality of third light sourcessmaller than the number of plurality of fourth light sources. In thiscase, a detailed method of generating the fourth depth information 412will be described in FIG. 7.

FIGS. 5A to 5C are conceptual diagrams illustrating the operation of amachine-learning-based model according to various embodiments of thepresent disclosure.

Referring to FIG. 5A, first depth information 511 on a first objectgenerated using a pattern of light reflected from the first object amongpatterns of light irradiated from a plurality of second light sourcesmay be input to the machine-learning-based model 510. In addition, whenthe first depth information 511 is input, the machine-learning-basedmodel 510 may output first depth information with reduced noise 512.

Hereinafter, for convenience of description, depth information generatedusing light irradiated from a plurality of fourth light sources greaterthan the number of the plurality of third light sources is referred toas reference depth information, and the first depth information 511transformed to converge to the reference depth information by themachine-learning-based model 510 is referred to as transformation depthinformation.

In one embodiment, the machine-learning-based model 510 may transformthe first depth information 511 to converge to the reference depthinformation. In this case, the machine-learning-based model 510 mayoutput the transformation depth information as the first depthinformation with reduced noise 512.

In one embodiment, the machine-learning-based model 510 may detect noisefrom the first depth information 511. For example, the machine-learningbased model 510 may detect noise from the first depth information 511and may output the first depth information with reduced noise 512 byreducing the detected noise.

For example, the machine-learning-based model 510 may detect a firstportion determined to be noise by comparing the transformation depthinformation and the first depth information 511. For example, themachine-learning-based model 510 may detect a portion in which thedifference between the transformation depth information and the firstdepth information 511 is equal to or larger than a predeterminedthreshold, as the first portion. The machine-learning-based model 510may output the first depth information with reduced noise 512 byreducing the noise detected from the first depth information 511.

Referring to FIG. 5B, the first depth information 511 and visibilityinformation 521 about a second object generated using a pattern of lightreflected from the second object among the patterns of light emittedfrom the plurality of second light sources may be input to amachine-learning-based model 520. When the second depth information 511and the visibility information 521 are input, the machine-learning-basedmodel 520 may output first depth information with reduced noise 513using the visibility information 521.

In one embodiment, the machine-learning-based model 520 may determinewhether there is a second part having a preset threshold or greater inthe visibility information 521. For example, when the second partexists, the machine-learning-based model 520 may determine a part oftransformation depth information corresponding to the second part andmay adjust the part corresponding to the second part based on thevisibility information. The machine-learning-based model 520 may outputthe adjusted transformation depth information as third depth informationwith reduced noise 522.

In another example, when no second part exists, themachine-learning-based model 520 may be trained to determine not toadjust the transformation depth information and to output thetransformation depth information as the third depth information withreduced noise 522.

In one embodiment, the machine-learning-based model 520 may detect athird portion that is determined to be noise although it is not actuallynoise from the first portion determined to be noise, by using thevisibility information 421. When the third portion is detected, themachine-learning-based model 520 may exclude the third portion from thefirst portion and may determine the first portion from which the thirdportion is excluded to be noise in the third depth information 511. Inaddition, when the third portion is not detected, themachine-learning-based model 520 may determine the first portion to bethe noise in the third depth information 511. The machine-learning-basedmodel 520 may output the third depth information with reduced noise 522by reducing the noise that is determined in the third depth information511.

Referring to FIG. 5C, a plurality of pieces of depth information 531,532, 533, and 534 about the first object generated using the pattern oflight reflected from the second object among the pattern lights emittedfrom the plurality of second light sources may be input to amachine-learning-based model 530. The machine-learning-based model 530may generate the first depth information 511 (refer to FIG. 5A), whichis representative depth information about the first object, using theplurality of pieces of depth information 531, 532, 533, and 534. Aftergenerating the first depth information 511, the machine-learning-basedmodel 510 may output the first depth information with reduced noise 512as described in FIG. 5A.

Further, although not shown, a plurality of pieces of image data aboutthe first object generated using the pattern of light reflected from thesecond object among the patterns of light emitted from the plurality ofsecond light sources may be input to the machine-learning-based model530. The machine-learning-based model 530 may generate the first depthinformation 511 (refer to FIG. 5A), which is representative depthinformation about the first object, using the plurality of pieces ofimage data. After generating the first depth information 511, themachine-learning-based model 510 may output the first depth informationwith reduced noise 512 as described in FIG. 5A.

In this manner, even when a relatively small number of pieces of imagedata are acquired to generate depth information, the printed circuitboard inspection apparatus 100 may remove noise, such as an unreceivedsignal or a peak signal, from the depth information on the component byusing machine-learning-based models 510, 520, and 530. Also, the printedcircuit board inspection apparatus 100 may generate the depthinformation on the component so that the lost shape can be restoredusing the machine-learning-based models 510, 520, and 530 even if arelatively small number of pieces of image data are obtained and thusinformation for generating the depth information is insufficient. Inaddition, the printed circuit board inspection apparatus 100 may notperform error restoration of the joint shape of the component whilecorrecting the 3D sharpness of the edges of the component as much aspossible, and may detect the shape of an additionally measured foreignmaterial without deteriorating the same.

FIG. 6 is a conceptual diagram illustrating a learning method of amachine-learning-based model according to various embodiments of thepresent disclosure.

In one embodiment, a machine-learning-based model 620 may include CNN,GAN, and the like. Hereinafter, a learning method of amachine-learning-based model will be described, focusing on GAN, whichcan perform image transformation using U-net. The machine-learning-basedmodel 620 may include a generator 621 and a separator 622.

In one embodiment, third depth information 611 on a second objectgenerated using a pattern of light reflected from the second objectamong patterns of light irradiated from a plurality of third lightsources is input to the generator 621. Fourth depth information 612 onthe second object generated using the pattern of light reflected fromthe second object among the patterns of light irradiated from aplurality of fourth light sources may be input to the separator 622.

In one embodiment, the generator 621 may generate transformed thirddepth information by transforming the third depth information 611 toconverge to the fourth depth information 612. The separator 622 mayseparate the transformed third depth information and the fourth depthinformation 612 by comparing the transformed third depth information andthe fourth depth information 612. The separator 622 may transmit resultsobtained by separating the transformed third depth information and thefourth depth information 612 to the generator 621. The generator 621 mayadjust a parameter for transformation of the third depth information 611according to the result received from the separator 622. This process isrepeated until the separator 622 cannot separate the transformed thirddepth information and the fourth depth information 612, so that thegenerator 621 may be trained to generate transformed third depthinformation by transforming the third depth information 611 to convergeto the fourth depth information 612.

Meanwhile, in the generator 621, the third depth information 611 and thefourth depth information 612 on any specific component form a pair. In acase in which any of the third depth information 611 and the fourthdepth information 612 has a poor quality (a case in which depthinformation of any one channel, such as a shadow area, a saturationarea, and an SNR, for each of at least one pixel is significantly lowerthan a predetermined reference value compared to other channels), thegenerator 621 may additionally perform a refinement operation to excludethe corresponding component data from learning data.

FIG. 7 is a diagram illustrating a method of acquiring depth informationused in training a machine-learning-based model according to variousembodiments of the present disclosure.

As described in FIGS. 4A to 4C, the fourth depth information 412 may begenerated in the printed circuit board inspection apparatus in which thethird depth information 411 is generated. For example, as illustrated inFIG. 7, it is assumed that the number of a plurality of third lightsources 710 is 4 and the number of a plurality of fourth light sourcesis 8. In this case, the processor of the printed circuit boardinspection apparatus 100 may control the plurality of third lightsources 710 to irradiate the component mounted on the printed circuitboard with a pattern of light, may generate the third depth information411 by using the pattern of light reflected from the component, and maythen move the plurality of third light sources 710 clockwise orcounterclockwise. The processor may control the plurality of third lightsources 710 moved clockwise or counterclockwise so as to irradiate thecomponent mounted on the printed circuit board with the pattern oflight, and may generate the fourth depth information 412 by using thepattern of light reflected from the component and the third depthinformation 411. Meanwhile, in FIG. 7, it has been described that thenumber of the plurality of third light sources 710 is 4, but this is forillustrative purposes only, and is not limited thereto. The number ofthe third light sources 710 may be one rather than plural. As such, thefourth depth information 412 may be generated in the printed circuitboard inspection apparatus including the plurality of third lightsources having a smaller number than the number of the plurality offourth light sources.

FIGS. 8A to 8C illustrate images of a component generated using depthinformation with reduced noise by a printed circuit board inspectionapparatus according to various embodiments of the present disclosure.

In one embodiment, the printed circuit board inspection apparatus 100may generate depth information on a component by using a pattern oflight reflected from the component among patterns of light irradiated onthe component mounted on the printed circuit board, from a plurality offirst light sources. In addition, the printed circuit board inspectionapparatus 100 may generate a 3D image of the component by using thegenerated depth information. However, noise may occur in multiplereflections of light irradiated on the printed circuit board or in theprocess of processing the received light by the image sensor. If thegenerated noise is not reduced, the quality of the 3D image of thecomponent generated by the printed circuit board inspection apparatus100 may be deteriorated, and accurate inspection of the mounting stateof the component may not be performed.

In one embodiment, the printed circuit board inspection apparatus 100may reduce noise from the depth information on the component by using amachine-learning-based model, and may generate the 3D image of thecomponent by using the depth information with reduced noise. Since the3D image generated using the depth information with reduced noise maymore accurately display the shape of the component, more accurateinspection of the mounting state of the component can be performed.

Referring to FIG. 8A, when the printed circuit board inspectionapparatus 100 generates the 3D image of the component by using the depthinformation as is without reducing noise, a shape of a connectionportion 810 (e.g., solder paste or the like) between the component andthe printed circuit board may be displayed as an abnormal shape in the3D image or displayed as having a hole, due to the noise. On the otherhand, when the printed circuit board inspection apparatus 100 reducesnoise in the depth information on the component by using themachine-learning-based model and generates the 3D image of the componentby using the depth information with reduced noise, a shape of aconnection portion 811 (e.g., solder paste or the like) between thecomponent and the printed circuit board may be more accurately displayedin the 3D image.

Referring to FIG. 8B, when the printed circuit board inspectionapparatus 100 generates the 3D image of the component by using the depthinformation as is without reducing noise, a shape of an edge 820 of thecomponent may be displayed as an abnormal shape in the 3D image due tothe noise. On the other hand, when the printed circuit board inspectionapparatus 100 reduces noise in the depth information on the component byusing the machine-learning-based model and generates the 3D image of thecomponent by using the depth information with reduced noise, a shape ofthe edge 821 of the component may be more accurately displayed in the 3Dimage.

Referring to FIG. 8C, when the printed circuit board inspectionapparatus 100 generates the 3D image of the component by using thegenerated depth information as is, an internal shape 830 of thecomponent may be displayed as an abnormal shape in the 3D image such ashaving a hole in the component due to the noise. On the other hand, whenthe printed circuit board inspection apparatus 100 reduces noise in thedepth information on the part by using the machine-learning based modeland generates the 3D image of the component by using the depthinformation with reduced noise, an internal shape 831 of the componentmay be more accurately displayed in the 3D image.

As described above, the printed circuit board inspection apparatus 100may display the shape of the component more accurately through the 3Dimage generated using the depth information with reduced noise, therebyperforming more accurate inspection of the mounting state of thecomponent.

While the foregoing methods have been described with respect toparticular embodiments, these methods may also be implemented ascomputer-readable codes on a computer-readable recording medium. Thecomputer-readable recoding medium includes any kind of data storagedevices that can be read by a computer system. Examples of thecomputer-readable recording medium includes ROM, RAM, CD-ROM, magnetictape, floppy disk, optical data storage device and the like. Also, thecomputer-readable recoding medium can be distributed to the computersystems which are connected through a network so that thecomputer-readable codes can be stored and executed in a distributionmanner. Further, the functional programs, codes and code segments forimplementing the foregoing embodiments can easily be inferred byprogrammers in the art to which the present disclosure pertains.

Although the technical spirit of the present disclosure has beendescribed by the examples described in some embodiments and illustratedin the accompanying drawings, it should be noted that varioussubstitutions, modifications, and changes can be made without departingfrom the scope of the present disclosure which can be understood bythose skilled in the art to which the present disclosure pertains. Inaddition, it should be noted that that such substitutions, modificationsand changes are intended to fall within the scope of the appendedclaims.

What is claimed is:
 1. A printed circuit board inspection apparatus thatinspects a mounting state of a component mounted on a printed circuitboard, the printed circuit board inspection apparatus comprising: aplurality of first light sources configured to irradiate the componentwith a pattern of light; an image sensor configured to receive a patternof light reflected from the component; a memory configured to store amachine-learning-based model, wherein when first depth information on afirst object generated using a pattern of light reflected from the firstobject among patterns of light irradiated from a plurality of secondlight sources is input into the machine-learning-based model, themachine-learning-based model outputs the first depth information withreduced noise; and a processor, wherein the processor is configured to:generate second depth information on the component by using the patternof light reflected from the component and received by the image sensor;input the second depth information into the machine-learning-basedmodel; obtain the second depth information with reduced noise from themachine-learning-based model; and inspect the mounting state of thecomponent by using the second depth information with reduced noise,wherein the machine-learning-based model is trained to output thirddepth information with reduced noise by using third depth information ona second object generated using a pattern of light reflected from thesecond object among the patterns of light irradiated from the pluralityof second light sources and fourth depth information on the secondobject generated using a pattern of light reflected from the secondobject among patterns of light irradiated from a plurality of thirdlight sources, and wherein the machine-learning-based model outputs thefirst depth information with reduced noise when the first depthinformation is input based on a training result.
 2. The printed circuitboard inspection apparatus of claim 1, wherein the number of theplurality of second light sources is the same as the number of theplurality of first light sources, and the number of the plurality ofthird light sources is larger than the number of the plurality of firstlight sources.
 3. The printed circuit board inspection apparatus ofclaim 1, wherein the machine-learning-based model comprises aconvolutional neural network (CNN) or a generative adversarial network(GAN).
 4. The printed circuit board inspection apparatus of claim 1,wherein the processor is configured to generate a three-dimensionalimage of the component by using the second depth information withreduced noise, and to inspect the mounting state of the component byusing the three-dimensional image of the component.
 5. The printedcircuit board inspection apparatus of claim 1, wherein, when visibilityinformation on the first object is further input into themachine-learning-based model, the machine-learning-based model outputsthe first depth information with reduced noise by using the visibilityinformation, wherein the visibility information includes a ratio of anamplitude of a brightness signal of image data to an average brightness,and wherein the machine-learning-based model is trained to determinewhether there is a portion having a preset threshold or greater in thevisibility information.
 6. The printed circuit board inspectionapparatus of claim 5, wherein the machine-learning-based model istrained to output third depth information with reduced noise using thirddepth information on a second object, which is generated using a patternof light reflected from the second object among the patterns of lightemitted from the plurality of second light sources, visibilityinformation on the second object, which is generated using the patternof light reflected from the second object among the patterns of lightemitted from the plurality of second light sources, and fourth depthinformation on the second object, which is generated using a pattern oflight reflected from the second object among patterns of light emittedfrom a plurality of third light sources, and to output the first depthinformation with reduced noise based on a training result when the firstdepth information and visibility information on the component are input.7. The printed circuit board inspection apparatus of claim 5, whereinthe processor generates visibility information on the component usingthe pattern of light which is reflected from the component and receivedby the image sensor, and further inputs the visibility information onthe component into the machine-learning-based model.
 8. A printedcircuit board inspection apparatus for inspecting a mounting state of acomponent mounted on a printed circuit board, the printed circuit boardinspection apparatus comprising: a plurality of first light sources toemit a pattern of light to a component; an image sensor to receive thepattern of light reflected from the component; a memory to store amachine-learning-based model, wherein when a plurality of pieces ofdepth information on a first object, which is generated using a patternof light reflected from the first object among patterns of light emittedfrom a plurality of second light sources, is input into themachine-learning-based model, the machine-learning-based model generatesfirst depth information and outputs the first depth information withreduced noise; and a processor, wherein the processor is configured to:generate a plurality of pieces of depth information on the componentusing the pattern of light which is reflected from the component andreceived by the image sensor; input the plurality of pieces of depthinformation on the component into the machine-learning-based model;obtain the second depth information with reduced noise from themachine-learning-based model, in which second depth information isgenerated by the machine-learning-based model based on the plurality ofpieces of depth information on the component; and inspect a mountingstate of the component using the second depth information with reducednoise, wherein the machine-learning-based model is trained to: generatethird depth information on a second object using a plurality of piecesof depth information on the second object generated using a pattern oflight reflected from the second object among the patterns of lightemitted from the plurality of second light sources; and output the thirddepth information with noise-reduced using the third depth informationand fourth depth information on the second object, which is generatedusing a pattern of light reflected from the second object among patternsof light emitted from a plurality of third light sources, and whereinthe machine-learning-based model generates the first depth informationand outputs the first depth information with reduced noise based on atraining result when the plurality of pieces of depth information on thefirst object is input.
 9. The printed circuit board inspection apparatusof claim 8, wherein the number of the plurality of second light sourcesis the same as the number of the plurality of first light sources, andthe number of the plurality of third light sources is larger than thenumber of the plurality of first light sources.
 10. A non-transitorycomputer-readable recording medium that records a program to beperformed on a computer, wherein the program comprises executableinstructions that cause, when executed by a processor, the processor toperform operations of: controlling a plurality of first light sources toirradiate a component mounted on a printed circuit board with a patternof light; generating first depth information on the component by usingthe pattern of light reflected from the component and received by animage sensor; inputting the first depth information into amachine-learning-based model; obtaining first depth information withreduced noise from the machine-learning-based model; and inspecting themounting state of the component by using the first depth informationwith reduced noise, and wherein, when first depth information on a firstobject generated using a pattern of light reflected from the firstobject among patterns of light irradiated from a plurality of secondlight sources is input, the machine-learning-based model outputs thefirst depth information with reduced noise, wherein themachine-learning-based model is trained to output third depthinformation with reduced noise by using third depth information on asecond object generated using the pattern of light reflected from thesecond object among the patterns of light irradiated from the pluralityof second light sources and fourth depth information on the secondobject generated using a pattern of light reflected from the secondobject among patterns of light irradiated from a plurality of thirdlight sources, and wherein the machine-learning-based model outputs thefirst depth information with reduced noise when the first depthinformation is input based on a training result.
 11. The non-transitorycomputer-readable recording medium of claim 10, wherein the number ofthe plurality of second light sources is the same as the number of theplurality of first light sources, and the number of the plurality ofthird light sources is larger than the number of the plurality of firstlight sources.
 12. The non-transitory computer-readable recording mediumof claim 10, wherein the machine-learning-based model comprises aConvolutional Neutral Network (CNN) or a Generative Adversarial Network(GAN).
 13. The non-transitory computer-readable recording medium ofclaim 10, wherein the executable instructions cause the processor tofurther perform operations of: generating a three-dimensional image ofthe component by using second depth information with reduced noise; andinspecting the mounting state of the component by using thethree-dimensional image of the component.
 14. The non-transitorycomputer-readable recording medium of claim 10, wherein themachine-learning-based model outputs the first depth information withreduced noise using visibility information on the first object when thevisibility information is further input, wherein the visibilityinformation includes a ratio of an amplitude of a brightness signal ofimage data to an average brightness, and wherein themachine-learning-based model is trained to determine whether there is aportion having a preset threshold or greater in the visibilityinformation.
 15. The non-transitory computer-readable recording mediumof claim 14, wherein the executable instructions cause the processor tofurther perform operations of: generating visibility information on thecomponent using the pattern of light which is reflected from thecomponent and received by the image sensor, and further inputting thevisibility information on the component into the machine-learning-basedmodel.
 16. A method of inspecting a mounting state of a component by aprinted circuit board inspection apparatus, the method comprising:controlling a plurality of first light sources to irradiate a componentmounted on a printed circuit board with a pattern of light; generatingfirst depth information on the component by using the pattern of lightreflected from the component and received by an image sensor; inputtingthe first depth information into a machine-learning-based model;obtaining the first depth information with reduced noise on thecomponent from the machine-learning-based model; and inspecting themounting state of the component by using the first depth informationwith reduced noise on the component, wherein, when first depthinformation on a first object generated using a pattern of lightreflected from the first object among patterns of light irradiated froma plurality of second light sources is input, the machine-learning-basedmodel outputs the first depth information with reduced noise, whereinthe machine-learning-based model is trained to output third depthinformation with reduced noise by using third depth information on asecond object generated using the pattern of light reflected from thesecond object among the patterns of light irradiated from the pluralityof second light sources and fourth depth information on the secondobject generated using a pattern of light reflected from the secondobject among patterns of light irradiated from a plurality of thirdlight sources, and wherein the machine-learning-based model outputs thefirst depth information with reduced noise when the first depthinformation is input based on a training result.